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A telephone connecting network is given, and with full information 
at all times about its state, routing policies are sought which minimize 
the expected number of attempted calls denied service in some finite interval. 
In this paper, the search is pursued as a mathematical problem in the con- 
text of a standard traffic model in terms of optimal control theory and dy- 
namic programming. Certain combinatorial properties of the network, earlier 
found to be the key to minimizing the loss, also turn out to be relevant 
here: they lead to policies which differ from optimal policies only in 
accepting all unblocked call attempts, and provide a ''practical" 
solution of the problem posed. In many cases, the policies found vindicate 
heuristic policies earlier conjectured to be optimal. 

I. INTRODUCTION AND SUMMARY 

We study the problem of optimally routing calls in a telephone 
connecting network during a finite time interval [0, t] over which the 
traffic intensity need not be constant. The present work reports on 
extensions of earlier results 1 on routing in telephone networks with 
constant traffic intensity; the principal novelty lies in the fact that 
whereas previously we minimized the probability of blocking* here 
we seek to minimize simply the expected total number of call attempts 
denied service in a given fixed time interval [0, t] on which the traffic 
intensity may vary. 

A traffic model, the same as that used in Ref. 1, is described (Sec- 
tions II to IV), and the problem is formulated mathematically in the 
manner of optimal control theory (Section V). The associated Hamilton- 
Jacobi equation is written and it is noted that this equation has a 
solution constructible in terms of functions satisfying nonlinear integral 

* Defined asymptotically as the stochastic limit, as t becomes large, of the 
fraction of attempted calls blocked or rejected in [0, £]. 
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equations derived from the principle of optimality, (Section VI). An 
isotony theorem, based on the same combinatorial properties as were 
used in Ref. 1 to minimize the loss fraction, then exposes the optimal 
policies to within rejection of unblocked attempted calls. That is, 
policies are arrived at which differ from optimal policies only in that 
the latter might reject some unblocked calls at some times (Sections 
VII and VIII) ; these policies are the same as those that were arrived 
at in minimizing the loss. 

II. STATES, EVENTS, AND ASSIGNMENTS 

The mathematical model of Ref. 1 will be used. The elements of 
this model separate naturally into combinatorial ones and probabilistic. 
The former arise from the structure of the connecting network and 
from the ways in which calls can be put up in it; the latter represent 
assumptions about the random traffic the network is to carry. The 
combinatorial and structural aspects are discussed in this section; 
terminology and notation for them are introduced. The probabilistic 
aspects are considered in a later section. 

A connecting network v is a quadruple v = (G, I, ft, S), where G 
is a graph depicting network structure, / is the set of nodes of G which 
are inlets, ft is the set of nodes of G that are outlets, and S is the set 
of permitted states. Variables x, y, z at the end of the alphabet denote 
states, while u and v (respectively) denote a typical inlet and a typical 
outlet. A state x can be thought of as a set of disjoint chains on G, each 
chain joining / to ft. Not every such set of chains represents a state: 
sets with wastefully circuitous chains may be excluded from S. It is 
possible that I = ft, that / fl ^ = " = null set, or that some inter- 
mediate condition obtain, depending on the "community of interest" 
aspects of the network v. 

The set S of states is partially ordered by inclusion ^, where X g y 
means that state x can be obtained from state y by removing zero 
or more calls. If x and y satisfy the same assignment of inlets to outlets, 
i.e., are such that all and only those inlets u t I are connected in x 
to outlets v t ft which are connected to the same v in y (though possibly 
by different routes), then we say that x and y are equivalent, written 
x ~ y. 

The set S of states determines another set 8 of events, either hangups 
(terminations of calls), successes (successful call attempts), or blocked 
or rejected calls (unsuccessful call attempts). The occurrence of an 
event in a state may lead to a new state obtained by adding or removing 
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a call iii progress, or it may, if it is a blocked call or one that is rejected, 
lead to no change of state. Not every event can occur in every state: 
naturally, only those calls can hang up in a state which are in progress 
in that state, and only those inlet-outlet pairs can ask for a connection 
between them in a state that are idle in that state. The notation e 
is used for a (general) event, h for a hangup, and c for an attempted 
call. If e can occur in x we write e t x. A call c t x is blocked in a state 
x if there is no y c S which covers x in the sense of the partial ordering 
^ and in which c is in progress. For h t x, x — h is the state obtained 
from x by performing the hangup h. 

We denote by A x the set of states that are immediately above x 
in the partial ordering ^ , and by B x the set of those that are immediately 
below. Thus, 

A x = {states accessible from x by adding a call} 

B z = {states accessible from x by a hangup}. 

For an event e t x, the set A ex is to consist of those states y 7* x to 
which the network might pass upon the occurrence of e in x. Thus, 
if e is a blocked call, A tz = 0; also 

U A hz = B z 

hex 

U A cx = A z . 

etx 
e not blocked in i 

The number of calls in progress in state x is denoted by \x\ . The 
number of call attempts c t x which are not blocked in x is denoted 
by s(x), for "successes in x." The functions |-| and s(-) defined on 
S play important roles in the stochastic process to be used for studying 
routing. In addition, we use 

/3 X = number of idle inlet-outlet pairs blocked in state x 

a z = number of idle inlet-outlet pairs in state x, 

and note that a = -\- s. 

It can be seen, further, that the set S of states is not merely partially 
ordered by ^, but also forms a semilattice, or a partially ordered 
system with intersections, with x f~\ y defined to be the state consisting 
of those calls and their respective routes which are common to both 
x and y. 

An assignment is a specification of what inlets should be connected 
to what outlets. The set A of assignments can be represented as the 
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set of all fixed-point-free correspondences from / to 0. The set A is 
partially ordered by inclusion, and there is a natural map y(-): S — > A 
which takes each state xtS into the assignment it realizes; the map 7(0 
is a semilattice homomorphism of S into A, since 

x ^ y implies y(x) ^ y(y), 

y(x f)y) ^ t(») fi y(v)- 

We denote by F x the set of calls that are free or idle in x, i.e., 

F x = {c: c is idle in x) = {y(y - x):yt A x ), 

where y — x is the state obtained from y by removing all the calls 
of re ^ y. 

III. PROBABILISTIC ASSUMPTIONS 

A Markov stochastic process x, taking values on S is used as a 
mathematical description of an operating connecting network subject 
to random traffic. Specifically, the Markov process of Ref. 1 will be 
used, with the modification that the calling-rate per idle inlet-outlet 
pair can depend on time. This model can be paraphrased in the informal 
terminology of "rates" by two simple assumptions: 

00 The hang-up rate per call in progress is unity. 
(ii) The calling-rate between an inlet and a distinct outlet, both 
idle at time u, is X(w) ^ 0. 

The transition probabilities of x t will be described after a discussion 
of system operation and routing. 

IV. ROUTING POLICIES 

It will be assumed here, as in Ref. 1, that attempted calls to busy 
terminals are rejected, and have no effect on the state of the network; 
similarly, blocked attempts to call an idle terminal are refused, with 
no change of state. Attempts to place a call are completed instantly 
with some choice of route, or are rejected, in accordance with some 
routing -policy. 

A routing policy over [0, t] will be described by a measurable matrix- 
valued function of time, denoted by R(u) = (r xv (u)), x, y e 8, ^ u ^ t, 
having the following properties and interpretation: for each x t S, 
let II, be the partition of A x induced by the relation ~ of "having the 
same calls up," or satisfying the same assignment of inlets to outlets; 
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it can be seen that n z consists of exactly the sets A BX for c t x, c not 
blocked in x; for each u z [0, t], Y z n x , r xu (u) for y z Y is a possibly- 
improper probability distribution over Y, (that is, it may not sum 
to unity over Y), 

r xz (u) = s(x) - X r xv (u) , 

VtAj 

and r xu (u) = in all other cases. 

The interpretation of the routing matrix R{u) is to be this: any 
Y zll x represents all the ways in which a particular call c (free and not 
blocked in x) could be completed when the network is in state x; for 
y z Y, r xu (u) is the chance that if call c is attempted in state x at time 
u, it will be completed by being routed through the network so as 
to take the sj'stem to state y. That is, we assume that if c is attempted 
in x at u, then with probability 

1 " £ r xy (u) (1) 

vt4«« 

it is rejected (even though it is not blocked), and with probability 
r xu {u) it is assigned the route which would change the state x to y, 
for y z A cx . The possibly improper distribution of probability 

{r xu (it), yz Y\ 

indicates how the calling-rate \{u) due to c at time u is to be spread over 
the possible ways of putting up the call c, while the improper part (1) 
is just the chance that it is rejected outright. 

It is to be noted that, as in Ref. 1, routing is carried out with perfect 
information about the current state of the network. The problem of 
optimal routing with only partial information is much more difficult 
(than the problem to be considered here), and it is not taken up. 

Alternatively, we may define the convex set C of all (routing) matrices 
R = ( r *v) such that r xu ^ 0, r xy = unless y t A T , and 

E r IV ^ 1, 

VtAcz 

for c z x not blocked in x, 



•„ = s(x) - E r xu , 



and describe the routing policies as measurable functions on [0, t] 
taking values in C. 

A routing policy R(-) with r xu (u) = or 1 is called a, fixed policy. 



„(w) = 
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V. FORMULATION OF THE PROBLEM 

For the purpose of denning a Markov stochastic process it is con- 
venient and customary to collect the probabilistic and operational 
assumptions made above in a time-dependent matrix Q(-) of transition 
rates. Indeed, each routing policy R(-) determines such a matrix func- 
tion, and so a process, according to the relationship Q = Q(R) given 
in detail by 

1, y*B x 

\(u)r xu (u) , y*A z 

- | x | - \(u) [s(x) - r xx ] , y = x 

0, otherwise. 

If the routing policy R(-) is used, the transition probability matrix 
P(u, t) = (p xv (u, t)), with 

Pxv(w. t) = Pr {x t = y | x u = x\ , 

will develop according to the backward Kolmogorov equation 

P(t, t) = I, Q = Q(R) 

^-P(u, t) = -Q(u)P(u, t), O^u^t. 

Oct 

In particular, if the system starts at with an initial probability 
distribution given by the column vector p(0), then its distribution p(u) 
at time u is [p(0)'P(0, u)}', which satisfies the equation p(u) = Q(u)'p(u). 
If the network is in state x at time u, the rate at which blocked or 
rejected calls are being generated is X(u)[r xx (u) + /3J. Thus, with 
r{u) = r(R(u)) the vector function {r xx (u), x c S}, the total expected 
number of calls denied service during [0, t] is just 

D = D(p(0), t) = f p(u)'[r(u) + (3}\(u) du. (2) 

Jo 

We may, therefore, state our routing problem thus: Minimize D 
subject to the conditions p(0) given, p(u) = Q(u)'p(u), Q = Q(R), 
R(u) e C, for u e [0, *]. 

Let us now view the | S |-dimensional probability vector p = p(u) 
as a "state-variable" whose "motion" is governed by the linear dif- 
ferential equation p(u) = Q(u)'p{u). The criterion D is linear in p(-) 
and the matrix entries of the control R(-) appear as coefficients in 
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the equation and in the criterion. The problem of minimizing D can be 
approached and solved by the now classical methods of the theory 
of optimal control. 

VI. THE HAMILTON-JACOBI EQUATION 

Let p, q be | S |-dimensional vector variables, and introduce the 
Hamiltonian function 

H(p, u, q, R) = \(u)p'((3 + /•) 

+ 2 PA 2 r h + X(u) 2 r iy q y ~ (X(u)r x + | x \)q x \. 

Let H* be the minimum of H for R t C, i.e., 
H*(p,u, q) = min [\{u)p'Hq + \(u)p'Rq - £p x [X(u)r, + | x \] Qx } , 

RtC x 

where H = (h xv ) is the "hangup matrix" such that h zu = 1 or ac- 
cording as y e B x or not. The Hamilton-Jacobi equation associated 
with the minimization of D above is just 

^ + H*(p, u, ^-) = 0. < u < I, p > 0. 

au V ap) - (3) 

V(p, - o. 

It follows from a lcnown theorem 2 of the theory of optimal control 
that if we can find a continuously differentiate solution V(p, u) of the 
Hamilton-Jacobi equation (3), then a control policy R(-) = (r IV (-)) 
such that by components 

R(w) -r~ (p, «) = min R -— (p, m), ^ m ^ / 
dp it i c dp 

is optimal. 

To find a solution of the Hamilton-Jacobi equation (3) let us con- 
sider the problem of starting the connecting system at a time u < t, 
and operating it until t so as to minimize the expected number of blocked 
calls over (u, t). We define, with t fixed, and u < t, 

E x (u) = expected number of blocked calls in (u, t) 
using an optimal policy, starting in state x. 

To solve the problem we note that two possibilities arise: Either an 
event occurs in (u, t), or else none does. In the latter case, the system 
stays in its initial state x throughout (u, t), and no calls are blocked 
during the interval. In the former case the first event e to occur does so 
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at some time epoch t t (u, t) and can lead to one of the states in A, z \J 
{x}. The minimum expected blocking to be suffered in the remaining 
interval (t, t) is just 

min {1 4- E x (t), min E v (t)} if e = c 

ycA c , 



E x . h (r) 



if e = h. 



With 



c x (u) = | x I + a z \(n) , 
C x {u) = f c x (v) dv, 



the probability density that the first event to occur does so at r, and 
is e, equals 



C(r)exp{-C,(r) + C x (u)} 



1 



C(r) ' 
X(r) 



A(r) ' 



e = h 



e = c. 



Hence, applying the "principle of optimality," we conclude that the 
vector function E(u),0 ^ u ^ t, satisfies the equation 

EM = f expl-CM + CM} 

.[£tf,(r) + X(r) £ min {1 + E m (r), min EMU dr. (4) 

yr«, Ctl VtAcx 

We now observe that if E(-) satisfies (4), then the scalar function 
V(p, u) = p'E(u) satisfies the Hamilton-Jacobi equation. This is of 
course not surprising since the equation for E(-) was obtained from the 
optimality principle. To see it we differentiate (4) with respect to u, 
obtaining 

%-em -(1*1 + Hu)a x )EM - Hem - x(tt)& 

era vi it, 

- X( M ) X) min {1 + EM, min EM) 

ctz VtAcm 

-[|*| + \(u)s(x)]em - Hem - x(«)A 



V tn, 



-\(u) 



min {1 + EM, min E y (u)\. 

utAti 



c not blocked in x 



ROUTING IN CONNECTING NETWORKS 2349 



Now note that 

2 mm {1 4- E\(u), mill E u (u)} 

etz vi Act 

r. not blocked in x 

2 min {(1 - 2 r zv )[EM + 1] + Z ^(u)} 

fix tftC Vi."»cr »i-4(f 

c not blocked in x 

= min { 2 (1 - Z 0[#» + 1] + Z r rv E y {u)} 

HtC ctz VtA,, vtA c , 

cnot blocked in i 

= min \[b{x) - T,r xy )[EM + 1] + Zr IV E y (u)\ 

HtC VtA. ytAm 

= min [r„[EM + 1] + T,r zv E v (u)}. 

HtC ytA. 

Therefore, 

|-tf x ( W ) + min [-(I x | + \(u)[s(x) - r xx \)EM 

+ HEM + X(u)[/3 X + r„] + Z^A(m)} - 0. 

»ifi, i/i. i« 

Now with V = p'E and dF/dp = E, r = r(R) = [r„ , x e S], p £ 0, 
^ + min {X(w)p'(/3 + r) - £ p x (| a: | + \(u)[s(x) - r ZI ])E x (u) 

+ Ep*(E + T,r ty )EM) =0. 

i vtB, VtA, 

This is the Hamilton-Jacobi equation. It follows that the minimum 
of D is achieved by a fixed policy, as could be expected on intuitive 
grounds. 

VII. ISOTONY THEOREM 

In Ref. 1 we introduced some combinatorial "monotone" properties 
of the partial ordering (S, ^) of states which (when present) provide 
an intuitive and straightforward description of the routing choices 
for accepted calls which minimize the loss probability. These properties 
are also relevant to minimizing the criterion D of (2). 

The properties in question can be paraphrased as follows : the relative 
merit of states vis a vis blocking is consistent or continuous, i.e., if 
a state x is "better" than another y, then the neighbors of x in ^ are 
in the same sense better than the corresponding neighbors of y. Spe- 
cifically, we deal in detail only with the weakest property used in Ref. 1, 
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and we say that a relation P on S has the weak monotone property 
if xPij implies 

(0 I x | = | y \, 

(ii) 3 m : B x «-> .#„ and z t B x implies zP/tz, 

(ms) 3 r: P„ -> F« and (a)ctF n ui (y imply 3 W c A {reU with wP0, 
(b) c, c' t F v , vc = vc' imply c = c'. 

We now prove the following isotony result : 

Theorem: If P is a relation on S having the weak monotone property, 
then xPy implies 

dV §V 

dp x = dp v 
Proof: Define recursively E x (0, u) = 0, 

E x (l, u) = p x [C x (t) - CM] exp [CM - C x {t)} 

= Pr {first & only event in (u, t) is a blocked call \x u = x], 

E z (n + 1, u) = f exp [CM - C x (r)} 

•[2», r) + &Mt)[EM t) + 1] 

VtB, 

+ X( T ) 2 min {1 + EM r), min E v (n, r)}] dr. 

CtX 1/t/tc. 

enot blocked in/ 

It follows easily that E(l, u) ^ E(u), and that E(n + 1, w) ^ #(n, w). 
Furthermore, standard methods 3 using the inequality 

| min y { — min (y { + e,) | ^ max | e, | 

show that the functions E x (n, ■) converge monotonely as n -* °o to 
the unique solution of (3). 

If now xPy, then & ^ & , c,(-) = c,(-), and so E z (l, u) ^ #„(1, «). 
Assume as a hypothesis of induction that xPy implies E x (n, u) ^ E u (n, u), 
^ u ^ t. Then with n and v as in the definition of the weak monotone 
property 

E,(n, u) ^ E^(n, u) for ztB x 
min {1 + E x (n,u), min E,(n,v)} 

^ min {1 + E y (n, u), min 2?.(n, u)} 

itAc, 
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E E,(n, u) + 0Xu)[E y (n, u) + 1] 

ttB, 

+ Hu) E min {1 + E v (n, u), min E,(n, u)} 

ctv tiAc, 

cnot blocked In 1/ 

^ E #.(*. w) + PMu)[E,(n, u) + 1] + X(u)(A, - ft)[5,(n, u) + 1] 

llH, 

- X(w)[A>,m) + 1] E i 

CCX 
c not blocked inx 

+ X(w) E mm U + 2?»(n, w), min £,(n, w)j . 



etx 
c not blocked in x 



It can be seen that with | X | the cardinality of a set X, 

P u — fi z ^ I {c t x: c i rng v, c not blocked in re) |, 

whence E s (n+1, u) ^ E u (n + 1, u). Since dV/dp = Z? and E(n, a) | #(*'), 
the theorem follows. 

VIII. THE NATURE OF THE OPTIMAL POLICIES 

Where it is applicable, the isotony theorem allows us to infer the 
optimal routes for accepted calls. Its relevance to the optimal policies 
for networks for which there is a relation P with the weak monotone 
property is this: Let c c x be a call that is not blocked in state x, so 
that A ex ?* 6, and suppose that there is at least one y t A cz such that 
yPz for every z t A ez . It follows from the isotony theorem that at any 
time m, such a y is at least as good a way of routing c (if c is attempted 
at u) as any other state of A rz . The only action which might con- 
ceivably be better in this situation than accepting c and routing it 
so as to take the system to y is rejecting c altogether. Such a rejection 
would be optimal if and only if 

dp z dp y ' 

for w's close to t, clearly, this is false. In these circumstances a policy 
that routes c in x so as to take the system to y can differ (so far as x 
and c are concerned) from an optimal policy only in the respect that 
the latter might reject c in x. 
In Ref. 1, the notation 

sup A„ 
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was used for the set 

{y.zt A ez implies yPz\ C\ A cz , 

whenever this set was nonempty. The set sup A cx consists precisely 

p 

of the possible states to which an optimal policy takes the system 

from state x if it accepts the attempted call c. 

The preceding observations are summarized in the Corollary: If P 

on S has the weak monotone -property then there exists an optimal policy 

R(-) such that ctx,y t A cx r xu (u) > 0, < u < t imply 

ytmp A ex . 
p 

The theory of routing for minimal D constructed here can be de- 
veloped in greater detail in the fashion of the optimal routing theorems 
of Section XVIII of Ref. 1 ; however, the isotony theorem and corollary 
embody the basic idea, and we shall leave the topic at this stage. 
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